Field of the Invention
The present invention relates to an image processing apparatus and image processing method of performing image segmentation.
Description of the Related Art
A segmentation method is used as a technique of dividing an image into a plurality of areas so that the attributes such as the colors, patterns, and brightnesses of these areas are the same. Since various kinds of image processing can be performed on these divided areas one by one, the operation amount can be reduced compared to a case in which an image is processed pixel by pixel.
Several methods have been proposed to increase the speed of the segmentation process. An example is R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S, Süsstrunk, “SLIC Superpixels,” tech. rep., EPFL, EPFL, 2010. (to be referred to as literature 1 hereinafter). Literature 1 has disclosed a method of dividing an image into a plurality of areas by clustering image data by using five-dimensional information containing color information (l, a, b) and coordinates (X, Y). This method described in literature 1 is called Simple Linear Iterative Clustering (SLIC). In this SLIC method, representative data as the centers of clusters are arranged in a matrix in an image. This representative data in the SLIC method has five-dimensional information containing color information (l, a, b) and coordinates (X, Y), and is also called a seed or cluster centroid. An area obtained by clustering is called a superpixel. The SLIC method is based on the k-means method as one clustering method, and clusters pixel data to the representative data arranged in a matrix. Then, each representative data is updated by using the five-dimensional information of the clustered pixel data, and clustering is performed again. The SLIC method obtains the segmentation result by repeating clustering and update once or more. The difference between the SLIC method and k-means method is that the range of pixel data to be clustered to the representative data is limited.
Also, C. Y. Ren and I. Reid. gSLIC: a real-time implementation of SLIC superpixel segmentation. University of Oxford, Department of Engineering, Technical Report, 2011. (to be referred to as literature 2 hereinafter) has disclosed a method of increasing the speed of the method of literature 1 by using a GPU (Graphic Processing Unit). Literature 2 has proposed a hierarchical clustering scheme in order to implement high-speed processing by the GPU, thereby implementing a high-speed segmentation process for a high-resolution image.
In addition, T. Maruyama, “Real-time K-Means Clustering for Color Images on Reconfigurable Hardware”, International Conference on Pattern Recognition, pp. 816-819, 2006 (to be referred to as literature 3 hereinafter) has disclosed a method of increasing the speed of the k-means process by improving a representative data initial value setting method. When processing a plurality of similar images in order, this method of literature 3 normally processes the first image, and processes the second and subsequent images by using representative data obtained by preceding images, thereby reducing the repeat count and increasing the processing speed.
When applying the speed increasing method as disclosed in literature 3 to a moving image, however, if a camera is moved, scenes are changed, or an image has largely changed, a phenomenon in which no pixel data is clustered to representative data occurs. This phenomenon is called a dead cluster, dead centroid, or empty cluster. In this specification, the phenomenon will be called a dead cluster. If this dead cluster forms, the number of areas reduces, and the segmentation accuracy decreases.